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Abstract 

We consider the issue of the slice invariance of refined topological string am- 
plitudes, which means that they are independent of the choice of the preferred 
direction of the refined topological vertex. We work out two examples. The first 
example is a geometric engineering of five-dimensional U{1) gauge theory with a 
matter. The slice invariance follows from a highly non-trivial combinatorial iden- 
tity which equates two known ways of computing the Xy genus of the Hilbert 
scheme of points on C^. The second example is concerned with the proposal that 
the superpolynomials of the colored Hopf link are obtained from a refinement of 
topological open string amplitudes. We provide a closed formula for the super- 
polynomial, which confirms the slice invariance when the Hopf link is colored with 
totally anti-symmetric representations. However, we observe a breakdown of the 
slice invariance for other representations. 



1 Introduction 



All genus topological string amplitudes on local toric Calabi-Yau 3-fold can be computed 
by a diagrammatic rule, in terms of the topological vertex [H El [3] ; 

C,Uq) = g^sA(gO , (1-1) 

V 

with three U{oo) representations, or partitions fi, v and A, which will be identified as 
Young diagrams throughout the present paper. In (11. II) sxi^jyx) is the (skew) Schur 
function and (^^^ means the substitution Xi := The dual partition is defined 

by the transpose of the corresponding Young diagram. The index k{v) is related to 
the (relative) framing of the topological vertex. The relation of Nekrasov's partition 
function [H O [6] to the topological string amplitudes motivates us to seek a refinement 
of the topological vertex [7] . We first proposed such a refinement in [8] by employing the 
(skew) Macdonald function P\/^{x] q,t). Later it has been improved by incorporating 
the framing factor [9] as follows: 

I I — 1 1^ I 

C^x^iq^t) = fAq,t)-'Px{t'';q,t)J2{l)^ iP,^ /.f{-t''\':t,q)P,,,{qh''-q,t) , 

(1.2) 

where qH'' etc. means the specialization xi := q^H^~'^ and fi, {q,t) is the framing factor. 
See |10] for more details on notations. A slightly different version of the refined topological 
vertex was introduced in [TT] : 

|HI^ + l|A|p 

CliV\qi' q^) = (jj-^ q~Px^iq2^] qi, ^2) 

( |»7| + |m|-|i^ 
- ) ' s,y,iq^\,ns^^,iq,'\^') , (1.3) 

which allows an interesting interpretation in terms of "unisotropic" plane partitions. This 
is also related to the statistical model of melting crystal [H |1T] . It has been proposed 
that a certain topological open string amplitude computed from C^JJx^\qi, Q'2) gives ho- 
mological invariants of the Hopf link [12]. The relation of the refined topological vertex 
and homological invariants is one of the main subjects of this paper. 

Though it is not manifest in the expression (11. ip . the topological vertex is symmetric 
under the cyclic permutation of three partitions. However, it seems impossible to keep 



1 



the cyclic symmetry for the refinements. Consequently, both C^x^^q, t) and C^^x (^'i' ^2) 
have a preferred direction, which gives us an issue of the choice of the preferred direction 
in the rule of diagrammatic computation by the refined topological vertex. In [10] we 
have proved that our refined topological vertex gives a building block of Nekrasov's 
partition function. To obtain Nekrasov's partition function we consider toric diagrams of 
the geometric engineering of A/" = 2 supersymmetric Yang-Mills theory [TH [T^ [TBI 17] . 
For such toric diagrams we required that the preferred direction at each vertex should be 
parallel each other. This requirement largely restricts a possible choice of the preferred 
direction. However, for a certain diagram there are more than one choice. In this paper 
we consider the possibility of changing the preferred direction. It is often the case that 
a summation over partitions remains for one choice of the preferred direction, while we 
can compute the summation completely for the other choice. Thus the independence of 
the preferred direction gives a kind of summation formula for partitions. In this paper 
we consider two examples of such summation formulae (see fll.4p and fll.71) below). The 
slice invariance is important not only for the consistency of the formalism but also for 
practical computations. 

The first example is a topological closed string amplitude that appears in the geo- 
metric engineering of five-dimensional U{1) theory with a matter. When we compute 
the closed string amplitude based on the refined topological vertex, there are alternative 
choices of the preferred direction in the toric diagram of the five- dimensional U{1) theory. 
The agreement of the amplitude requires a highly non-trivial combinatorial identity; 



where the light hand side is a summation over partitions A. a{s) and i{s) in the product 
are the arm length and the leg length that are defined by the corresponding Young 
diagram. The identity (ll.4p was discussed before in [T71 UHl HO]- It is known that this 
amplitude gives the generating function of the equivariant Xy genus of the Hilbert scheme 
of points on C^. The validity of fll.41) has a deep geometrical meaning. Namely the left 
hand side of (11. 4p comes from a computation of the generating function by the localization 
for a toric action on the Hilbert scheme which is induced by {zi,Z2) (e*^^2;i, 6**^^2:2) on 
C^, where the fixed points are labeled by Young diagrams. On the other hand a formula 
that computes the Xy genus of the Hilbert scheme of a surface S from the Xy genus of 




(1.4) 
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the underlying surface S has been estabhshed in [19]. There is also a proposal of more 
general formula of the elliptic genus based on string theory or the orbifold conformal field 
theory [20] . The right hand side of (11. 4p can be derived, if we apply these formulae to the 
equivariant Xy genus of C^. Thus the identity (11.41) equates these two ways of computing 
the equivariant Xy genus of the Hilbert scheme, one by the localization principle, the 
other from the corresponding Xy genus of the underlying manifold. 

The second example comes from a refinement of topological open string amplitudes, 
which is concerned with the relation of the refined topological vertex to homological 
refinements of the link polynomial. According to the proposal in [12], the homological 
invariant of the Hopf link colored with representations A and /i is proportional to 



^a,m(?17?2,Q) = X](-Q)''''g|"'' " Zr^v{q^,q2)Zr,{q2,qi)sxiqi'^q2^)s^,{qi''q2 

■n 

(1.5) 

where 

Uq..qi) = n (l - ^?'^"^rO" = n - <il''^\tY . (1.6) 

Note that the summation over the partition rj remains in the definition of ^a,/x- Therefore 
the fact that the homological invariants are polynomials in Q is not clear at all in the 
above proposal. In this paper we work out the summation in (II. 5p . when both A and /i 
are totally antisymmetric representations, (A,/i) = (1^, 1*) and prove 



" 1=1 



z!.'!((^g^t)^ = {-irt-'^'^'~''>er{tnes{^I^Qq'''>t^,rn | | ( 1 " Qq^t'^^^ ) , (1-7) 



where (g, t) = {qi^, q2^) and • means the trivial representation. er{x) and es(x, y) are the 
elementary symmetric functions. The fundamental summation formula on the space of 
symmetric functions is the Cauchy formula [21]. However, we cannot apply the Cauchy 
formula directly to (II. 5p . We will heavily rely on many properties of the Macdonald 
functions and the Macdonald operator to be introduced in section 4, so that we can 
use the Cauchy formula to perform the summation over partitions. Though our proof 
of the formula (11.70 can be made without referring to the refined topological vertex, 
it is closely related to a change of the preferred direction as has been argued by Taki 
|22] . (see section 3 for the explicit form of his conjecture). The above formula agrees 
with his conjecture (up to normalization), when both A and fi are totally anti-symmetric 
representations. This means that we can show the invariance under the change of the 
preferred direction in this case. However, for general representations Taki's conjecture 
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should be appropriately modified as discussed in section 6. Recently the homological 
invariants for the pair (l'^, 1) is constructed by Yonezawa [23j by the method of matrix 
factorization. If we apply his result to the Hopf link, we find a complete agreement to 
our formula (11.71) up to an overall normalization. 

The paper is organized as follows; in section 2 we consider the toric diagram of the ge- 
ometric engineering of five-dimensional U{1) theory and show that the slice independence 
of the refined closed topological string amplitude gives the combinatorial identity ( 11. 4p . 
It turns out that the validity of (11.41) is equivalent to the agreement of two known ways 
of computing the Xy genus of the Hilbert scheme of points on C^. In section 3 we review 
the relation of the homological invariants of the colored Hopf link and the refined open 
topological string amphtudes following [12]. We also introduce the conjecture presented 
in [22] and explain how it is related to the problem of changing the preferred direction. 
Section 4 is the main part of the paper. We prove the formula (ll.7p by making use of the 
Macdonald operator on the space of the symmetric functions. In section 5 by using the 
formula we provide a general formula for the superpolynomial of the Hopf link colored 
with any pair (A,/i) = (1'^, 1*) of totally anti-symmetric representations. We also show 
that homological invariants obtained from (II. 5p are actually two variable Laurent polyno- 
mials in q and t with positive integer coefficients. Some examples of the superpolynomial 
for representations other than the totally anti-symmetric ones are worked out in section 
6. By the base change between the Schur functions and the Macdonald functions, we 
can replace one of the anti-symmetric representations (I'', 1**) with any representation A. 
We compute the superpotential explicitly when A is the symmetric representation or the 
hook representation. We also argue some general structure of the superpolynomial of 
the Hopf link colored with (A, I''). In appendix combinatorial identities coming from the 
slice invariance of the toric diagram of f/(l)^ theory are presented. 

The following notations are used through this article, q and t are formal parameters 
otherwise stated. Let A be a Young diagram, i.e., a partition A = (Ai,A2,---)) "which 
is a sequence of non-negative integers such that Aj > Aj+i and |A| = J^i'^i < -^^ 
denotes its conjugate (dual) diagram. £(A) = A^ is the length and |A| = is the 

weight. For each square s = in A, a(s) := Aj — j and i{s) := AJ — i are the 

arm length and the leg length, respectivelj{l]. The notations for the symmetric functions 
in X = {xi,X2, ■ ■ ■) are the Macdonald function Px{x;q,t), the Schur function sx{x) 
and the elementary symmetric function ex{x), respectively. The algebra of symmetric 
functions A can be identified as a polynomial ring in the power sums {pn{x)}'^^i with 

""^In [H] the definitions of tlie arm lengtli and the leg length are exchanged. 
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Pn{x) = Yli^i ^7- symmetric functions in this article are treated as polynomials in the 
power sum symmetric functions {pi,P2, ■ ■ ■ )■ For \t^^\ < 1, x = q'^t^ means Xi = q'^H^~''. 
By definition a specialization of the symmetric functions is an algebra homomorphism 
p : A R. For example one may take R = <C. Since we treat the symmetric functions as 
polynomials in p„, we can define a specialization by giving the images p{pn) of the power 
sums under p. For example the specialization of the symmetric functions with x = qH'' 
is defined by 

Pn(gV) := y - + — = y + (1.8) 

^ t2 —t 2 ^ t2 —t 2 

1=1 1=1 

which is independent of as long as > ^(A). In this specialization the algebra 
R = Q(g, t) is the field of rational functions of q and t. We also note that for Pn{x, y) : = 

Pn{.x) +Pniy), 

n_Tn 

p„(cgV, = c" y (g"^' - l)^^^-) + 4 —, c,LeC. (1.9) 

t2 — t 2 



1=1 



An involution t acting on the power sum function p„(x) is defined as Pn{Lx) := —pn{x). 

The definition of the g-integer in this articlcl is \n\„ := -r- = g^~" tt. In 

q — q^^ 1 — q^ 

terms of [n]\g = [n]g[n — 1]^ ■ ■ ■ [l]g, the g-binomial coefficient is 



1 _ ^2{n-k+i) 



n^^^- (1-10) 

i=l ^ 



For n, G N both the g-integer and the g-binomial coefficient are polynomials in g with 
positive integer coefficients. Finally, we often use Q := vQ with v := {q/t)'^. 



2 Five-dimensional U{1) theory with extra matter 

Changing the preferred direction in the diagrammatic computation of the refined topo- 
logical vertex often gives a highly non-trivial combinatorial equality that involves a sum- 
mation over partitions. In this section we provide an example which shows a curious 
connection of the change of the preferred direction and a combinatorial identity in enu- 
merative geometry. 

^ We follow the standard convention in knot theory. The definition [n]q := g"/^ — 5^"^^ is also used 
in the literatures. 
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By using the Macdonald function Px{x;q,t), the refined topological vertices are de- 
fined as [TOl 



C"^\(g,t):=C^v,v^"(t,g)(-l)l^l+l^l+IH (2.1) 
= Pav (-g"; t, q) J2 Pu-/a- i-t^'q"; t, q) P,,M^tP- q, (g, t) , 

C/'^(g, t) := C^\(g, (g, t)~' /. (g, ty' , (2.2) 

with the framing factor 

(2.3) 



fxiq,t):= n 

(i,i)eA 

The lower and the upper indices correspond to the incoming and the outgoing represen- 
tations, respectively, and the edges of the topological vertex are ordered clockwise. The 
middle index A is the representation for the preferred direction. 




• — *- 




A V A 



X" A 



Figure 1: Change of the preferred direction, which is indicated by the white arrow. 

As has been discussed in [171 [10], the refined topological string amplitude for the 
diagrams in Fig. 1 gives the generating function of the equivariant Xy genus of the Hilbert 
scheme Hilb'^(C^) of n points on C^. These diagrams are one- loop diagrams where we 
identify two external vertical edges and arise from the geometric engineering of five- 
dimensional U{1) gauge theory with adjoint matter [7]. In the left diagram the preferred 
direction is along the internal line, while it is along the external lines in the right diagram. 

The gluing rule of the refined topological vertex gives the amplitude for the left 
diagram 

ZL:=J2Q^"^^^^^C.x'iq,t)C'\iq,t) 
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J2 Q^'^A^'^PxitP; q, t)Pxv (-g^; t, q)P.{qH^; q, t)P,. (-t^'g^; t, q). (2.4) 



Here • stands for the trivial representation, i.e., the zero Young diagram (0, 0, ■ ■ ■ ). On 
the other hand the right diagram gives us the following partition function; 



= gl'^l Al^lp,v/,^v {~iqP- 1, q)Px/a, {tP- 1, g)PAv/.,v {-qP; t, q)P,,„, {if- 1, q)v 



(2.5) 

The computation of Zl is made by the Cauchy formula for the Macdonald function 
Y Px/t,ix; q, t)Pxv/^v{y; t, q) = Ilo{x, y) ^ P/,v/^v {y; t, q)P^/^{x; g, t), (2.6) 



A 

where 

" ' " (2.7) 



Uo{-x, y) := exp i - ^ ^Pn{.x)pn{y) \ = JJ(1 - Xiyj). 



Note that for c G C, Uq^cx, y) = Ilo{x, cy) and for the involution i{pn) = —Pn, no(6x, y) = 
Ilo{x, Ly) = Ilo{x,y)~^. We also use the following adding formula 

Px/ti{x; q, t)Pf,/^{y; g, t) = Px/v{x, y; g, t). (2.8) 
From the formula of the principal specialization 

^' ^) = n (-g- t, g) = n (2.9) 

sex ^ sex ^ 

and the Cauchy formula (12.61) for = z/ = •, we have 



ZL = y^ no(-QgV, t^^qp) TT , , ,^iw.^ r^-r^r^- (2.10) 

uv ^-i ) ^ ^ 11 _ qa{s)^e{s)+l^^l _ g~a{s)-l^-e{s)^ ^ ^ 

If we define the perturbative part by := <5'''''^..''(9> ^)'^*%(9> ^) = ^o(-Qt^ g^), 
which is independent of A, then the instanton part Z^£^^ := Zl/Z^^^^^ is 

^zinst^Y^TT -lA ^ (yg t i <^g t 



7 



where Q = vQ. Here we use ((2.10) in pXi]) 

= Ilo{-v-^QqHP, r"qP)/Uo{-v-^QtP, qP). (2.12) 

The computation of is more involved and we have to employ the trace formul3[§(see 
(B.26) in [8]), which is obtained by successively using (12. 6p . 

^/.v/^v (x; t, q)Px/.{y, q, t)Px^/,- {z; t, q)P^/r,{w; q, t)^!^!/?-!''^''^'^"'''' 

k>0 

= exp i - ^ ^ ^^^^ (a''pn{y)Pn{z) + ^Pn{y)Pn{x) 

L n>0 ^ 

+7>n(w^)Pn(x) + ^Pn{w)p^{z) " | , (2.13) 

with c := a'-f/l36. Using this trace formula, we can compute the partition function Z^i as 
follows; if we put (a,/5, 7,5) = {A,v~^,Q,v) and {x,y,z,w) = {—Lq^.t^.—qP^it^), then 
c = gA. Hence from fl213D 



^ Ho(t^-AcV)Ho(t^-gc^g^) 1 

^ A-L no(t/', -t;c'=+igp)Ho(t'', -v^d'+^qp) 1 - c^'+i 



fe>0 

' E-^f^^".^^"^"^^:^^"^^"-c")^ (2.14) 
^nl-c"V fit -rtVot -o-t) \ ^ ' 



exp 



n>0 



As before we define the perturbative part by 



^pcrt 
R 



ZniA = 0) = exp I - 5^ f^" ^ ^ I . (2.15) 



Then the instanton part Z'^^^ := Zr/Z^'^^ is 



^ 1 ^nl-c"ft2 -t-2Vo2 -0-2)1 



n>0 



This is obtained by applying the automorphism ujq.t{pn) ■= (^1)" ^ XJ^n Pn on y and w in (B.26) 
of 0. 
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with u := (qt)^ . If the topological partition function is independent of the choice of the 
preferred direction in our one-loop diagram, Zf^^^ = Z'^'^^ and we should have 



^a(s)^£(s)+l I _ -a(s)-lf-e{s) 

A sex 



exp 



1 An (l-ty)(l-g-y) ^ 



. n>0 



A"y" (l-t")(l-g 



Thus the slice invariance requires the highly non-trivial identity fl2.17p . One can check 
the validity of ( 12.17P for special cases y = and y = 1. When y = it reduces to 

I] n (1 _ ga(s)i^W+l)(l _ q-a{s)-H-m) = """"P 1 5Z ~ (1 _ tn)(l _ f ' (^-IS) 

which was proved by Nakajima and Yoshioka [6]. The proof in [6J is geometric. Namely 
we can see that the right hand side is the generating function of the Hilbert series of 
Hilb"(C2) as follows; 



y A"ch ijO(Hiib"(c2), o)= Y\ - — ^— - = exp <^ y — . 

n=0 k,e>0 ^ ^ ' Ln>0 ^ ^ ' 



(2.19) 

On the other hand the left hand side arises from a computation of the generating function 
by the localization theorem for toric action, where the fixed points of the toric action are 
labeled by partitions. We also have a combinatorial proof based on the Cauchy formula 
(12. 6p for fi = u = •. The formula of the principal specialization (12.90 implies the desired 
identity. For y = 1 the conjecture is simply 

5: Al^l = exp l^i^U fid -AT'. (2.20) 

A ln>0 J n=l 

which is nothing but the generating function of the number of partitions. Physically y = 
e"™ in terms of the mass m of the adjoint matter. Thus the equality (I2.17P interpolates 
the massless theory {J\f = 4 theory) and the infinitely massive theory (A/" = 2 theory) . 

As has been pointed out in |24J, the right hand side of (12.171) is identified as the 
generating function of the orbifold equivariant Xy genera of the symmetric product 
Sym"(C^) := (C^)"/S'„. The Hirzebruch Xy genus of a manifold M is defined by 

Xy{M) := ^(-l)Vdimif''(M, A^T^M) , (2.21) 
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and we will consider the equivariant version of (12.211) where we use the equivariant 
cohomology i/^(M, APT*M). Since 



C[z^,Z2l (p = 0) 

C[Zi, Z2]dZi © C[Zi, Z2]dZ2, {p = 1] 

C[zi, Z2\dzi A dz2-, {p = 2) 



H^,{C^,APT*C'') = 0, (g^O) 
the generating function of the equivariant Xy genus of is given by 

(l-yti)(l-yt2) 
(l-tl)(l-t2) 



J2t'li{l-y{t, + t2)+y\t2) 



(2.22) 
(2.23) 

(2.24) 



k,e=o 



A formula of the elliptic genera of the symmetric product Sym"M was proposed in |20j . 
which gives the elliptic genera of Sym"M in terms of those of the underlying Kahler 
manifold M, see also [25]. The formula tells us that 



oo oo ^ 

J2 ."i?//o.b(Sym"M; g) = n n 713^ 



n=0 1=1 £,m 

where c(m, £) are the coefficients of the elliptic genus of M; 



(2.25) 



(2.26) 



ml 



Since the Xy genus is obtained from the elliptic genus by Ell(M; y, q = 0) = y '^'"2'^' X-yiM), 
from DMVV formula (12:251) we find 



X-,(Sym (C )) = [[ 11 (1 _ Am^^-i^fc^^^) (1 _ 



(2.27) 



n>0 

which means 



m>l kl>0 



log I 5^ A"X-,(Sym"(C2)) [ = l^n™^n(™-l)^nfc^n£(^ _ ^n^n)(^ _ yU^. 

l,n>0 j n>lrn>lk/>0 



1 A" 



.V (l-l/"q)(l-l/n^) 

^nl-A"?/" (1 -t'^)(l -t^) 



(2.28) 



On the other hand the localization theorem tells us that the left hand side is the equivari- 
ant Xy genera of the Hilbert scheme Hilb"(C^) of points on C^. Thus the slice invariance 
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fl2.17p geometrically means that the equivariant Xy genera of Hilb"(C^) which gives a 
resolution of the symmetric product agrees with the orbifold equivariant Xy genera of 
Sym"(C^). One may wonder how they agree in mathematically rigorous sense. For 
equivariant Euler character the agreement was proved in [6]. Furthermore, the following 
formula of the generating function of the Xy genera of the Hilbert scheme of a smooth 
projective surface S was established in [19j; 

f ^"X-,(Hilb»(S)) = e.Jf:^^'^\ . (2.29) 

ri=0 lm=l ^ ) 

This formula for the equivariant case would give us a proof of (12.171) . 

3 Refined topological vertex and homological link in- 
variants 

In [26] it was argued that homological link invariants are related to a refinement of the 
BPS state counting in topological open string theory. Based on this proposal, in [12] a 
conjecture on homological link invariants of the Hopf link from the refined topological 
vertex has been provided. Let us review their proposal briefly. Let L be an oriented link 
in with I components. We consider homological invariants of L whose components are 
colored by representations Ri, - ■ ■ ,Re of the Lie algebra sl{N). This means that we have 
a doubly graded homology theory " '^'^^/.^ whose graded Poincare polynomial is 

V.m;Ru-,RM,t) = J2 q^t^dim^f )^^--'^^(L) . (3.1) 

Substitution of t = — 1 gives the unnormalized link polynomial Psi{n);Ri,-- ,Ri{<i) = 
P5[(7V);iji,-..,_R^(q, — 1)- The normalized invariants Pbi{n);Ri,- - ,Ri{^) are obtained by divid- 
ing by the invariants of the unknot. However, in the following we only consider the 
unnormalized one. 

The physical interpretation of homological invariants as the BPS state counting leads 
to a prediction on the dependence of the link homologies on the rank — 1. It has been 
conjectured [271 |28] that there exists a "superpolynomial" P/^^ ... ij^(a, q, t) which is a 
rational function in three variables such that 

^0[(7V);i?i, -,K,(q,t) = P/ji,-,R.,(a = q^,q,t) . (3.2) 
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We can reproduce the polynomial invariants of the colored Hopf link from topological 
open string amplitudes on the conifold with appropriate brane configuration [291 SOI EH 
[32l 133] . Since they are computed by the method of topological vertex, it is natural to 
expect that a refined version of the topological vertex gives a homological version of 
the Hopf link invariants. Based on the refined topological vertex introduced in [11], the 
superpolynomial for the colored Hopf link was proposed in |12j ; 

P.,„(a,q.t) = (-l)!*!"! (q- ( ^^■f"""^' , (3.3) 

V V ^2/ \q2j Z.,,(gi,g2,<3) 

where 



^A,M(9i,g2,<5) = J^(-<5)''''g| ql Zrjv{qi,q2)Zr^{q2,qi)sx{qi'^q2^)s^{qi'^q2'''' 

V 

(3.4) 

The natural variables of symmetric functions {qi,q2) are related to the variables of the 
superpolynomial by 

7^2 = q, v^=-tq, Q = -ta"^. (3.5) 
The factor in (13. 4p is given bjfl 

4(^2, go = n (i - ^2^''^"^rO" = n (i - ^"^^i^^o" ' (3-6) 

and, therefore, related to the following specialization of the Macdonald function; 

P^(tP;g,t) =rlll^"lP J](l-g-«Wt-^W-l)-' . (3.7) 

seA 

Namely if we identify (gi, ^2) = {q^^,t^^), then we find 

ZxAqi,q2,Q) = J2{-Q)^"^p,Aq';t,q)P,{t';q,t)sx{qn^)s,{qnn . (3.8) 

Note that this expression is manifestly symmetric in A and fi. But due to the summation 
over 7], it is not clear at all that Zx^^^ is a polynomial in Q. Furthermore, since the 
superpolynomial Pa,/^ is defined by (13.11) and (13.21) . we would like to check that, when 
a = q^, the superpolynomials derived from (13.81) are in fact Laurent polynomials in q 
and t with positive integer coefficients. Thus it is desirable to have a closed formula of 
Zx,^l without a summation over partitions. 



■^The definitions of the arm length a{s) and the leg length £{s) in [T2] are exchanged, compared with 
the standard ones, for example in [2T]. 
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A — V 

> 

slicing 



+ 



Figure 2: Slicing and Flop; the preferred direction is indicated by the bold line I . 

The configuration that leads to the proposal fl3.3p is the toric diagram of the resolved 
conifold with two Lagrangian brane insertions (see Fig. 2). Two branes are inserted 
on the same side of external edges. But they are on the different vertices. In [12] 
the refined topological string amplitude for this diagram was computed by choosing the 
internal line as the preferred direction. This was the reason why the summation over 
the partition t], which was attached to the internal line, remained in the proposal (13.31) 
for the superpolynomial. As has been pointed out by Taki [22], if we assume the slice 
invariance that means the partition function computed by the refined topological vertex 
is independent of the choice of the preferred direction, we may have a formula of Vx,fj, in 
a closed form without a summation over partitions. Namely we first change the preferred 
direction from the internal line to the external lines as shown in Fig. 2. Then we make a 
fiop operation to move two Lagrangian branes to the same vertex. When we compute the 
topological string amplitude based on the final diagram, we can perform the summation 
over the partitions of the internal line and obtain 

-1 lQi\ I Qi\ ^-i(K(A)+K(/.)) 



Mli'MliV.Q-'Jf^lS) n (l-<?"'9r*"9'i""'*') . (3.9) 

Since the fiop operation in the computation of the refined topological vertex was worked 
out completely in [10l[22], we only have to assume the slice independence to derive (13. 9p . 
In other words the validity of (13. 9p is equivalent to the slice invariance. In the next 
section we prove (13. 9p agrees to the original proposal (13. 3p up to an overall normalization, 
if both A and fi are totally antisymmetric representations. The proof relies on various 
properties of the Macdonald functions and is independent of the computation of the 
refined topological vertex. This means the slice invariance holds in this case. However, 
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in section 6 we will see that we need a few correction terms to the formula (13.91) for 
general representations. 



4 Macdonald operator and summation over parti- 
tions 

In this section we will work out the summation over the partitions t] in (13.81) . when both 
A and fi are totally anti-symmetric representations. The most fundamental summation 
formula on the space of symmetric functions is the Cauchy formula 

y2 /pip\ P>^(^' ^' ^)Px{y; q, t) = n(x, y; q, t) := exp <^ V -Pn{x)pn{y) 

(4.1) 

where the scalar product is given by 

(PxlPx),, n 1 _ • (4-2) 

sex ^ 

If we try to apply it to (13. 8p . the problem is the existence of two Schur functions whose 
specialization depends on the partition rj. Thus our strategy is to "remove" these Schur 
functions from the summand. As we will see below this is possible for totally anti- 
symmetric representations. The starting point is the observation that for totally anti- 
symmetric representation A = (1^) both the Schur function sx{x) and the Macdonald 
function Px{x]q,t) coincide with the elementary symmetric function er{x). Thus we 
can use a trick of replacing one of the Schur functions with the Macdonald function to 
make use of the following remarkable symmetry of the specialization of the Macdonald 
functions [21] (Ch. VI.6); 

Pxit^; q, t)P,{qH''- q, t) = P.it^- q, t)Px{qn^^- q, t). (4.3) 

We have 

Zir,is (Q; q, t) = J2(-Q) ^'^Pv^ (^'^ ^' ^' (^'^'^ t)es{q''n 

= eritn $^(-g)l''IP,v (g"; t, q)P,{q^'''h''- q, t)e,(g''t'') . (4.4) 

V 

Thus we have succeeded in getting rid of one of the Schur functions from the summand. 
To eliminate the remaining elementary symmetric function in (14.41) . we introduce the 
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Macdonald operator on the space of symmetric functions [21]. The Macdonald func- 
tions are characterized by the property that they are simultaneous eigen-f unctions of the 
Macdonald operator. The Macdonald operator in variables x = {xi, ■ ■ ■ ,xn) is given 
by 

D'N =tll f^^-' (4.5) 

i=l 1 

where Tq^ is the g-shift operator defined by Tg^^f^x) = f{qx). It is known that the 
Macdonald polynomials are the eigen-functions of Z)]^; 

N 

D],Px{x- q, t) = 4,a^a(x; g, t), ejv.A := ^"^"^ ^'^'^ ■ (^.6) 

1=1 

More generally for non-negative integer r, we define the higher Macdonald operators i5]y 
by := 1 and 

D^,:=f(^-^y^J2U'-^UT..^^ (l<r<iV) (4.7) 

\I\=r iSl J i£l 

where the sum is over all r-element subsets / of {1, 2, ■ ■ ■ , A^}. We set D]y := for r > N. 



N 



Let Djy := '^r=o^N^^ ^ then the Macdonald polynomial is the eigen-f unction of D 

DnP\{x; q, t) = Px{x] q, t)eN,x, 

N N 

eN,x ■■= + zq'H''-') =: ^ Fe,(g^-t^-^). (4.8) 

1=1 ■r=0 

Therefore, Z)]v simultaneously diagonalized by the Macdonald polynomials 

D'i^Pxix; q, t) = Px{x- q, t)e,(g^'t^-^), (4.9) 

and commute with each other; [-D}y, -D|^] = 0, on the space of the symmetric functions 
in variables. 

Since Dat is not compatible with the restriction of the variables defined by xat = 0, 
we need to modify it so that we can take the limit N ^ oo [SU [35]. Let us define H and 

which act on a; = (xi, 0:2, ■ ■ ■ ) by the limits H := limAr_,oo and := limAr^oo H^, 
where 



.n>0 ) r>0 



AT; Z :— Zt 2 . 
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min(r,A'') r—s 



H^^ = t^'^-^> E n^-^^' r = 0,1,2,.... (4.10) 



s=Q i=l 



Here we use ^„>o -2" nr=i(^* ~ 1) ^ = ^^P {^n>Q^~^T^^ ~ ^"') ^Z''^} define a new 
parameter z by scaling z. Then 



^-^ n 1 



n>0 



I, n>0 \i=l /J 



exp 



\ = J2'^^r^^'^'y (4.11) 



n 

n>0 J r>0 



Since this is independent of as long as it is larger than the length i{X) of the partition, 
we have 

HnPx{x; q, t) = Px{x; q, t)EN,x, 

H'^Pxix; q, t) = Px{x- q, t)e,(gV). (4.12) 

Now by (I4.12P we can eliminate the elementary symmetric function in (14.41) and apply 
the Cauchy formula (14.11) as follows; 

(Q; q, t) = Crit") j^^P,{t-P; q, t)Pr,{q^''kP; q, t)e.(g''tO 

= er{n}_^ (H {x)P^{x-q,t))\^^^_, 

= er{tn (h%x)U{x, Qq^'"hP; q, t)) , (4.13) 

V / x=t-P 

where we have also used the following formula proved in [8] 

P,v(-t^^g^;t,g) = .[.^f^^ P,(g- V^; g, t). (4.14) 

The final step is the evaluation of H^{x)Ii{x, Qq^^^'h'^; q, t)\x=t-p- For this purpose we 
have to compute the action of the shift operator Tg^^,. on the Cauchy kernel Il{x,y; q,t). 
Since Tg^x.Pn = ((g" - l)^;^ + Pn) for the power sum p„ = J2i ^i^ 

—- -— = exp <^ > x^ pn{y) ) . 4.15 

^{x,y;q,t) n J 
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Hence, 



U{x,y]q,t) 



{ Yl - ^^ri ) 



. n>0 jG/ 
t" - 1 



. n>0 iG/ 



When X = t ^, since txj — = for j = i + 1, only / = {A^, — 1, ■ ■ ■ ,N — r + 1} 
contributes to the summation in i\A.7\i . Therefore, 



i=l 



r r—s 



Hi,u.,-, = i<i-«)- E n n (4.17) 

s=0 i=l j=l 



Then we have 
Proposition. 



D',nM)j^ = exp h: l^r<".i.p„(,. n 1^ „ 



n>0 ^ j=l 

H;,U{x y,q,t)\ ^ ^_iyt~rir-me^^y^ r") + O(t^), (4.18) 
n(t-^,?/;g,t) 

wt/i T.r>oi-^y(^r{x,y) := exp{-X;„>oPn(a;,l/)2:"/n}. 
Proo/. By ( KWf . 

= exp [Y. r(^+^Vn(l/,rO ,(4.19) 



. n>0 



we get the first equation. We also have 
HMli{x,y]q,t)\^=t-f 



Ii{t-P,y-q,t) 



= ti^--y E -p i E ^'^^'^rPniy, t-n [ n fF fl i^- (4-20) 

s=0 Ln>0 j i=l j=l 

Then by the following lemma, we obtain the second equation. q 
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Lemma. 

r ( CO 



(i-r 



s=0 



. n=l 



n 



X 



r+l^ 



k=l 



(4.21) 



SJ 



i=l 



Proof. Let us fix /c and consider the coefficient of on the left hand side. By expanding 
the exponential we see that it consists of the terms with t~^^ {0 < i < k). Hence we have 
terms proportional to 



^(-r^)^t^^(^-i) , {o<i<k) . 



(4.22) 



However by the formula ([21], Chap. L2, Example 3); 



r-l 



jj(i+f2) = j2z't^'^'~'^ 



1=0 



(4.23) 



we see that it vanishes as long as < ^ < < r. Finally among the coefficients of x^ only 
the term with survives by the same reasoning as above. Hence we may compute it 
with 



(.-1) 



s=0 



n=l 



(4.24) 



by omitting "1" in the left hand side of (14.211) . Comparing the coefficient of x^ in the 
relation of pn and Cr, we see the coefficient of x^' is 



r(-l)'^e,^(-t-")'t^'('-i 



s=0 



r-l 



(4.25) 



i=Q 



which completes the proof. 

The following is the main result of this section. 
Proposition. For \t\ < 1, 

^pi!^' = (-l)^ri^(^-)e.(tOe.(Qg(^')t'', t-) (l - Qg^t^+i 
= (-l)'-+^t-5'-(^-i)-|^(^-i)e,(Qt^ t-'')es{Qq^^^hP, t""), 



□ 



(4.26) 



where er{x) and es{x,y) are the elementary symmetric functions in variables x = {xi,X2,Xs, 
and two sets of variables {xi,X2, - ■ ■ ,yi,y2, ■ ' ') > respectively. 
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\x=t-P 



Proof. By taking the large limit of the proposition proved above, we see 

Zi.,p(Q;g,t) = e,(tO (^if^(x)n(x, Qg^^'^V; g, t)' 

= {-iyt-'^'^'-^hr{tP)Ii{t-P, Qq^^'h"- g, t)e,{Qq^^'\t-P) . (4.27) 

Finally the proof is completed by noting 
n(r^Qg(i^V;g,t) 



Ii{t~P,QtP-q,t) 



iVl^.(Q; g, t) = n (i - Qq^t''^^) , (4-28) 
1=1 



where A^A.^t is the denominator factor of Nekrasov's partition function f l2.12p . which sat- 
isfies ((2.12) in [ID]) 

, U(QqHP,q-H-P;q,t) 
NxJQ; g, t) = ^ . 4.29 

□ 

The condition \t\ < 1 in the above proposition can be eliminated By the 

proposition it is clear that for totally antisymmetric representations the superpolyno- 
mial P;v^^(a, q, t) is actually a polynomial in Q and consequently in a, which is not 
manifest in the proposal (13. 3p . We can also prove that it is a polynomial in g and t with 
positive integer coefficients when Q = t^ with G N. 



5 Homological invariants for totally ant i- symmetric 
representations 

To accommodate the standard convention of link polynomials, we make the following 
change of variablejfl; 

Vi = cC\ v^=-t"'q"', g = -ta-2, (5.1) 
so that g := (g/t)^ Q = a-2. Then 

P,,,(a,q,t) = (-a)l^l+H(-t)I^IH ^^'^ff^'^| . (5.2) 

^Note that compared with [12], we have changed the translation rule by q ^ q^"'^ and t t^^. 
Accordingly the identification or the t-grading of Q is also changed. 
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Substituting the formula (14.261) . we obtain 

r 

P(i.),(P)(a,q,t) = (-a)'-+^(-t)'--(-l)^q^(-i)e,(tOe.(Qg(^^)t'',t-0 J] (l " Q^''^'] 

i=l 
r 

= Si-'-\-ty'(i'^'-^^er{tP)es{q^^^hP, a^t"^) JJ (q2(^-i) - a^) . (5.3) 

4=1 

This is a rather simple closed formula of the superpolynomial for the Hopf link colored 
with totally anti-symmetric representations (l'',l*). Note that apart from the overall 
factor (— t)'" '^, t-dependence only appears in the specialization x = q^^^^H^ of es{x,y). By 
putting t = — 1 (or q = t) and taking the limit a — we find 

limP(i.),(p)(a,q,-l) ~ a-'--^q^(^-i)+^(^-i)e,(gOe,(g(^'')+0 • (5-4) 

a^O 

We note that there is an additional factor q''(''~i)+*(*~^) in the normalization of (15. 2p . 
compared with the large behavior of the polynomial invariants of the colored Hopf 
link. When s = the superpolynomial becomes independent of t. The formula (15.31) 
gives 

„2(i-l) „2 ^ ^2(i-l) „2 

va,u-. q. t) = a-q^'"-" n = n • <^-^' 

i=i ^ ^ H 

This agrees with the superpoljTiomial of the unknot with a totally ant i- symmetric repre- 
sentation in [121 [22] • For a = q^ it reproduces the homological invariants of the unknot, 
which has been proposed to give the Hilbert series of the Grassmannian Gr{N,r) |28j . 

To provide more examples of our formula, let us look at the case where one of the 
representations is the fundamental representation. Then the formula ( 15. 3p is evaluated 
to be 

k 

□(a, q, t) = a-'-\-t)'^e,{tne,{q^'%^, aH~n J] (q'^"' - a^) 

i=l 

= a--H^qi^(-^) TT ^^-^ ( t-s!!:!^ + .(5.6) 

q^-q-^ V q-q-' q-q v 



When /c = 1, we find 



'Po, q, t) = t — t 



A _2 1 - q-^ + a^-q^ 
q — q^ \ q — q^ q — q^ 

(aH2q2 - a2(t2q4 + tW - q' + 1) + t'q' - q' + l) 



a2(l - q2)2 

(5.7) 
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Up to the factor (— t)^^ or a shift of t-grading by —1, this result agrees with [12], where 
it was shown that for a = this superpolynomial gives rise to the Khovanov-Rozansky 
invariants of the Hopf hnk. 

It is remarkable that for any k the substitution of a = reduces the superpolynomial 
V (ifc) Q(a, q, t) to a polynomial in q=^-^ and t=^^; 



k 2i—2—2N 

□(q, t) = t'^N,-N^H^-l) Yl ( t-^q^' 



- 1 / n ,„ o q^ - q 



1=1 



+ 



^2N _ q2fc 



:q-2V^'"'^(-t)' 



N 
k 



q — q ^ q — q~^ 

{[kW^'-H-' + [N - kUci'"^^') , 



(5.8) 



-I q 



which gives homological invariants of the Hopf link with representations (l'^,n) of sl{N). 
Note that this expression vanishes for k > N as it should be for the totally antisymmetric 
representation (1^) of sl{N). We have factorized the standard normalization factor q"^^ 
for the Hopf link. Since both the g-integer and the g-binomial coefficient are polynomials 
with positive integer coefficients, (15.81) shows that 'P(ifc), □(q, t) is a Laurent polynomial 
in q and t with positive integer coefficients. Up to an overall factor q'^('^~^)(— t)'^, (15.81) 
agrees to the recent result by Yonezawa by matrix factorization [23] . 

More generally we can show that the superpolynomial P(ir) (■^^•((a, q, t) always gives 
a polynomial in q^^,t^^, when we substitute a = q^. In fact after the substitution we 
have 



P(l.),(P)(q, t) = (_l)r(s-l)qJV(r-s)+s(s-l)+ir(r-l)^r.. J-l- , 



- 1 



q* -q- 



r-s —sN+s(s—l)+r{r—l) 



-t)'-q 



N 
r 



(5.9) 



Hence what we have to show is that es{q^^''h^,c^^t ^) is a polynomial. Since the ele- 
mentary symmetric function is a polynomial in the power sums pi, ■ ■ ■ ,Ps, let us look 
at Pkiq^^'hP, q^^t-P). We find 



^2kr _ 1 _,2fciV _ _,2fcr 

p^iq^'-h", q^^t-O = (tq)-^'^4 \ + ^ ^ 



qfc _ q fc 



qfc _ q-fe 



^■-2fc„fc(r-2) [^^]q ^k(N+r) [k{N r)]q 



(5.10) 



which shows that they are polynomials. Thus we see that e<j(g'-^'^-'t^, q^^t"^) is a polyno- 
mial in q^^, t^^. 
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6 Case of other representations 



Using the fact that the Macdonald functions and the Schur functions are related by a base 
change of the space of symmetric functions, we can compute the homological invariants 
for other representations than totally anti-symmetric representations. The base change 
is rather complicated in general. But let us illustrate the method of computation in the 
simplest examples. 



6.1 (A,/i) = ((2),(1-'^)) case 
When |A| = 2, the base change is 



S(i2)(x) 



1 - qt 
1 



P(i2)(x;g,t) 



(6.1) 



By transforming the Schur functions to the Macdonald functions, we can apply a similar 
way of computation to the case of totally anti-symmetric representations. We obtain 

t-q 



1 

1 - qt 
1 



" ^(2),(i«)(Q;<?.*) ■ 

Z.,,{Q;q,t) 

g(i2)^(i3)(Q;g,t) 

. Z.,.{Q;q,t) . 

and hence 

P(2),(P)(a,q,t) = a^+VV-i) ( (.(2)(t'' 



P(2) (tP; q, t)iV(2),.(Q; g, t)e,(Qg(2)iP^ ^-p) 
P(i2)(t^; q, t)iV(i2),.(Q; q, t)e,{Qq^^\p , t"^) 

(6.2)' 



t-q 
1 - gt' 



i^i2~,{tP))N^2)^,{Q-q,t)e,{Qq^^hP,t-P) 



t-q 
1-qt 



,2){tP)N^,2),,{Q;q,t)esiQq^'"hP,t-P) 



(6.3) 



The terms proportional to give a discrepancy to the conjecture (13.91) . It holds only 
for totally anti-symmetric representations. As we argued in section 3, this means that 
the slice invariance of open topological string amplitudes is broken, if the representation 
attached to the topological brane is not totally anti-symmetric. The above formula for 
s = provides the following superpolynomial for the unknot colored by the symmetric 
representation; 

IN 



P(2),.(a,q,t)=a-VV' 



[q- q i)(q2 - q 2 



— (tV(a^-q^)+q^-l) • (6.4) 
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When s = 1, we find the following superpolynomial for the^Hopf link colored by the 
symmetric representation and the fundamental representatior 



^(2),(i)(a, q,t) = at^ 



q 



[q- q~^)(q^ - q"^) 



— 



+ 



,q- - ; )(i - a-^)(i - a-t-q-)(t-v^ + 

q " — t ^q ^ „-2\/i „-2 2\/j.-2 -2/ , 3\ , ~ q* 



1 - t-2q-4 

,-2 +-2^-2 



q - q 1 



a-Vq-' 



1 - t-2q-4 

fa2-l 



(l-a-^)(l-a-V)(t-V'(q + q 



q - q 



[q — q ^)2(q2 — q ^) 



((a^ - q^)(a^ - q^) + t-^q-^(q^ - l)(q^ + l)^(a^ - q^) 
-t-^q-^(q^ - l)^(q^ + l)(a^ - q^) + t-«q-«(q^ - l)^(q^ + 1)) 



(6.5) 



Note that there is a cancellation of the factor (l — qt) = (1 — t^^q^^) in the denominator. 
Finally by substituting a = q^ we can eliminate the remaining factors in the denominator 
to get a polynomial in q^^ and t^^. We believe this is a highly non-trivial consistency 
check of our formula (14.261) . 

6.2 (A,^) = ((21), (1-^)) case 

The second example is the hook representation A = (21) with |A| = 3. In this case we 

(t-g)(l + g) (t-g)(t'-g) 



1 






1-qH 
1 





'l-qt){l-qH) 
{t-q){t+l) 

1 - qt^ 
1 



P{3){x;q,t) 
[x;q,t) 



(21) 



P(i3)(x;g,t) 



(6.6) 



By the same manner as above we obtain the superpolynomial 



^{2i),{p)(a, q,t) 



1-qt' 

+ ^^^^t S(ia)(t0iV(i3),.(g; q, t)e,{Qq^''%P , r") 



qt' 



(6.7) 



^Examples of the superpolynomials provided in appendix of 112! are only for totally antisymmetric 
representations and this is a new example of the superpolynomial for the colored Hopf link. 
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When s = the superpolynomial for the unknot is 

/^(2i),.(a, q, tj = a t q -r-(t q (a - q j + q - Ij . (b. 

(q — q ^)"'(q'^ — Q ) 

When s = 1 the superpolynomial is 
^(21), □(a, q,t) 
:aV 



2.6 (l-a-^)(l-a-V) 



(q - q-i)(q2 - q-2)(q3 _ q- 



;q+ q- - ^^^^^t^^^^ - a-t-q-)(t-q- + t^q^ 



^_ (a^ l)(a^ (t6p6(a,q)+tV^P4(a,q)+tV^P2(a,q) + q-^Po(a,q)) , 

(q- q~^)'^(q'^ - q-^) 

(6.9) 



where 

P6(a,q) = (a2-q^)(a2-q«) , 

P4(a, q) = (a2 - q4)(q2 - l)(q2 + l)(q2 + q + l)(q2 - q + 1) , (6.10) 
P2(a, q) = -(a^ - q^ - q4)(q2 _ l)2(q2 + q + i)(q2 _ q + i) ^ 
Po(a, q) = (q2 - lf{c,^ + q + l)(q2 _ q + i) . 

We can again confirm a cancellation of the factor (1 — qt^) = (1 — t^^q^^) in the de- 
nominator and the fact that the superpolynomial reduces to a polynomial in q''^^ and t^^ 
when a = q^. 

6.3 {\^) = (A, 1") case 

Generalizing the above computations, we can show the "finiteness" of the superpolyno- 
mial for (A, yu) = (A, 1*). As a Laurent polynomial in a the degree of PA,(i'')(a, q, t) ranges 
from — |A| — s to A + s. When we substitute a = q^ with a fixed A^, 'PA,(i'')(q^5 q, t) 
vanishes if < max(|A|, s). By the relation (15. 2p these properties follow from the propo- 
sitions below. Since the set of the Macdonald functions {P\{x; q, t)}\\\=d is a basis of the 
symmetric functions of homogeneous degree d, we can write the Schur function s\{x) by 
the Macdonald functions as 

^A(a;)= Yl Ux,M^t)P^{x-q,t), (6.11) 

M 
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where U\^^{q,t) is a rational function. Then we have 
Proposition. For \t\ < 1, 



ZxAQ-, q,t) ^ ^ U,,,{q,t)g^P,{vQtP,t-^;q,t)e,{vQqn^,t- 



Z,,,{Q]q,t) 

9x--=\{{~l)q''^'h'^-'''\ (6.12) 

seA 

which is a polynomial of degree |A| + s in Q. 
Proof. From flG.lip 

ZxAQ:<l,t) = Y.^-Qf'^Pr,^{q'-,t,q)P,{tP-q,t)s^{qH^)sAq'n = Yl UxJ,,is{Q; q,t), 

(6.13) 

where 

Z,,,s{Q;q,t) := 5^(-Q)l^lp,v(g''; t, g)P,(t^; g, t)P^(g''t''; g, t)sp(g^tO- (6-14) 
Then by 1^ and fITOD . 

z,,AQ;<i,t) = 5Z7|^§^A(^"'';?>^)^M(^';?>^)n(?'^^';9>^)^p(?'^0- (e.is) 

Since Sp(g^t^) = e,(g^t''), by fimj) and fHl8|l . 

(t;g)l''l 



(Q; g, t) = PA'; q, t) /p\p^ Pviq't"; q, tWP,{x- q 



A\ 



x=t-P- 



= PA'; q, t)H\x, vQq^tP- q, t) At-p 

= PA'; q, t)AAt-'^eAQq''t', r^Mt'', vQq^t'; q, t). (6.16) 

But by Km and ((5.20) in [TO]) 

PA'; q, t)N,,.ivQ; q, t) = PAQt', ^A q, tA^^^fA^ t), (6.i7) 

with the framing factor fA^^) defined by (12.31) . we have 

"^'2 = AAt-'^v-\'^\pAQt', tA q, AAQq^p. tAfA, t). (e.is) 
^;»w; q, 

□ 
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Note that, forNeZ and > 



1 -t 



-Nn 



N 



t2 - 1-2 




(6.19) 



i=l 



i=l 



Hence, P^{qH'' ^t^^^P) is the Macdonald polynomial in variables {q'^^t'i^'^}i<i<N and 
vanishes for C.{X) < N < i{fi). Therefore when vQ = t^, we have 
Proposition. If N E Z>o and \t\ < 1, 



which vanishes for < N < max(|A|, s). 

Proof If < iV < P^{tP,t'^~P;q,t) = 0. If < N < s, Pis{qHf,t'^~P;q,t) = 0. 
Similarly, if < < s, Pis{tP,t'^-P;q,t) = 0. U s < N < P^{q^^'HP,t-^-P;q,t) = 
0. Hence the middle and the right hand side of (16.201) vanish for < < max(|yu|, s). 
On the other hand, if A^ > max(|/x|, s), by the finite A^ version of (14. 3p . 



Pp {tP, t-^'-P- q, t)P,{q''tP, t-^-^; g, t) = P.it" , t"^"''; g, t)P,s {q^t^, r"^-"- g, t), (6.21) 



we have the proposition. □ 
The condition \t\ < 1 in the last two propositions can be eliminated [36] . 
Finally we should make a remark on the fact that the transition function [/^^^(g, t) in 

(16. lip is a rational function in g and t. It is not obvious that the formulae in the above 

propositions are in fact polynomials in g and t. However, we have checked the following 

conjecture up to d = 7 by direct calculation; 

Conjecture. For \X\ = d, J2ii\i_i\=d^\fj-9fj-(^~^)iJ;i' '^^ ^ polynomial of degree d{d — l)/2 
in q and of degree d{d — l)/2 in . 

Under the above conjecture, if A^ G Z>o, 



ZA,p(^-^t^;g,t) 
Z,,,{v-H^;q,t) 



91^ J2 U,,,g,P,{t''+'',t-P;q,t)P,s{q>^t''^P,t-^-q,t) 



— I A 



g,sP,s{t''^P,t-''-q,t) J2 U,,,g,P,{q'^''h''+P,t-P;q,t), (6.20) 




is a polynomial in g and t. 
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Appendix: Five-dimensional U{1)^ theory 

Here we generalize the argument in section 2 to the five-dimensional U{1)^ theory. We 
compare two partition functions associated with the diagrams in Fig. [3l Since the differ- 
ence is just the choice of the preferred directions, we expect that they coincide. 
Let 

N 2N 

{Aa}a=l «=1 
N 

= E n ^' ^)^-^— {q'^-'t"; q, t) 

2N 

xPa..v (-g^; t, g)PA.._.v {-t^--\'>- 1, q) J] QL'"', (A.l) 

a=l 

with Ao = A2Ar and ctq = <^2N- The summations over the partitions with odd suffices 
-^1) -^3) -^5) ■ ■ ■ ) are performed by the Cauchy formula (12. 6p for fi = u = •. From the 
specialization formula: 

Px{t^-q,t)P^.{-q^;t,q) = (A.2) 

Nxx{l;q,t) 



with (12.121) we have 



= E n N ( ,. no(g^---t^ -Q,._,t^-^/). (A.3) 
1 iLi ^^A2«A2« i-L; q, t) 
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A, 



— • 
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Figure 3: 

port 



Zr 



^' Q2N 
^2N 



If we separate out the perturbative part := ZL{Q2a = 0) = nQ=i ^o(t^^ —Q2a~i<l' 



then from (I212D . Zi'^'^* := Zl/Z^'^ is 



^inst 



V2 



'2a 



(A.4) 



On the other hand, let 

N 



2N 



{A<,}a=l 
N 



a=l 



1] ^^A2.-lV-2.-l^(-/;^.9)^A2c«-2/<X2.-l(^'';9,^) 
{Aq} a=l o-2a-i 



AT 



2Ar 



X 



E ^A2.vM.v (-g^; t, g).PA2._./.2. (i"; 9, ^) n t^i'^^^-^i-i'^^-i n 



0=1 
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with Ao = A2Ar and do = cr2N- The following trace formula is useful for calculating Zr. 
Proposition. [To] For G N, let x° = x""*"^^ 's be sets of variables, Xa's be the Young 

diagrams, Xq = X2N, Ca,a+1 = Ca+2N,a+l+2N G C, Ca,l3 ■= nf=aCa,a+l and C ■= Ci^l+2N = 

Ill=iCa,a+i- If\c\ < 1, then 

N 2N 



N N-1 

= n n n no(-^", c,„,„«,«c"x-«^«) 

n>0 a=l f3=0 

= ^^p(-E^T3^(EE^2„,2„+2,-^lPn(x^"K(-x^^^^^ (A.6) 



n>0 l,«=l /3=0 



Let (c4„_3,4„_2,C4a_2,4a-lC4a-l,4aC4a,4a+l) := {v,Q2a-l,V ^ , Q2a) with V := {q/t)^ and 
4a-3 ^4a-2 ^4a-l ^4a\ — /+p _ , ,+p _^p\ i-U^^ U^.r^ ^— Vt'^^ 



[X "X 



, x^") := {tf, -tqP, Ltf, -qP), then we have c = Uali 



/3 



C2a,2a+2/3+l = J]^ <5q+7, (A. 7) 



l-(-l)'" 
2 

7=0 

with QQ,4.2Af := Qa and 

2a 2a+2/3+l ("l)'^ 

[q2 — q 2 j \J;2 — t 2j 

Therefore, from flA.6p . it follows that 

1 1 I g{{Q"J;<i",t 



1 ^ n 1 - C" I (t2 - t 2 ) (02 - 2 ) II 

with 



n>0 



2Af 2N-1 P 

^({Q J; t) := E E (-1 A^"'^""^ n ^ 



a+7 

a=l /3=0 7=0 
2Af 2Af-l /3 



a=l /3=0 7=0 

2N 

E - - QcM- ■ ■ (1 - Qa+27V-l) •■■)))• (A.IO) 



Here := f *■ ""^^"^^Qa- 
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If we separate out the perturbative part 



:= Zn{Q2. = 0) = exp|-Vl ^T,^ ^^f"^ 1 , (A.ll) 



then Zi^^t := Zr/Z'^j^'^ is 



n>0 



with 



AT / 27V-1 (3 2N (3 

a=l \ /3=0 7=0 13=1 7=0 

AT 

= - ^"'Q2a-l)Q2a(l - Q2a+l(l " Q2a+2(- " " (1 " Q2a+2iV-l) " " " )))• (A.13) 

a=l 

Since Zf^^'^^ = Zf^'^^, the shce invariance Z™*^* = Z)^^* is equivalent to the the following 
conjecture: 
Conjecture. 

\- TT ^|A2ai ^A2cA2c+2(Q2a+i; g,^) _ } 1 1 / /({<5a}; ^'^ 

with c := Yli=i Qa, Qa+2N ■= Qa and 

N /2N-1 (3 2N (3 

fHQ.y, <i,t):=Y,[Y. (-i)"" n ^^..+7 + - E(-i)'' n ^2«-i+7 

a=l \ /3=0 7=0 ^ /3=1 7=0 

AT 

It \ 

?2a+3 ■ ■ ■ (1 — Q2a+2Ar-l) "■■)))• 



(A. 14) 



( 1 <52q-1 ] <52a(l — Q2q!+i(1 — Q2a+2(1 — Ql 



(A.15) 



Computer calculations support this conjecture. We have checked this by Maple for iV < 3 
and |A2a| < 5. 

For = 1, Ql = Q and Q2 = A, then Z^ and Zr are equal to those in section 2 and 
the above conjecture reduces to (]2.17p . 

When N = 2 and Q3 = Q4 = 0, the conjecture reduces to 



n(l-r)(l-g") f 4-^2 A^A,A(l;g,t) 



. n>0 
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From the specialization formula fl2.9p or flA.2p . we find that this coincides with the identity 



of the slice invariance discussed in subsection 5.9 of [TT]. It is interesting that the slice 
independence implies the same condition in spite of the difference of our refinement of 
the topological vertex and the refined topological vertex proposed in [TT]. Thus the above 
conjecture generalizes that in [11] . 
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